An Online Learning Approach to Buying and Selling Demand Response

We adopt the perspective of an aggregator, which seeks to coordinate its purchase of demand reductions from a fixed group of residential electricity customers, with its sale of the aggregate demand reduction in a two-settlement wholesale energy market. The aggregator procures reductions in demand by offering its customers a uniform price for reductions in consumption relative to their predetermined baselines. Prior to its realization of the aggregate demand reduction, the aggregator must also determine how much energy to sell into the two-settlement energy market. In the day-ahead market, the aggregator commits to a forward contract, which calls for the delivery of energy in the real-time market. The underlying aggregate demand curve, which relates the aggregate demand reduction to the aggregator’s offered price, is assumed to be affine and subject to unobservable, random shocks. Assuming that both the parameters of the demand curve and the distribution of the random shocks are initially unknown to the aggregator, we investigate the extent to which the aggregator might dynamically adapt its offered prices and forward contracts to maximize its expected profit over a time window of T days. Specifically, we design a dynamic pricing and contract offering pol✩This work was supported in part by NSF grant ECCS-1351621, NSF grant IIP1632124, US DoE under the CERTS initiative, and the Simons Institute for the Theory of Computing. This work builds on our preliminary results, presented at the IFAC 2017 World Congress (Khezeli et al., 2017). The current manuscript differs significantly from the conference version in terms of new results, formal proofs, and more detailed technical discussions. We thank Weixuan Lin for many helpful discussions, and his assistance in the proof of Lemma 2. Email addresses: [email protected] (Kia Khezeli), [email protected] (Eilyan Bitar) icy that resolves the aggregator’s need to learn the unknown demand model with its desire to maximize its cumulative expected profit over time. The proposed pricing policy is proven to be asymptotically optimal — exhibiting a regret over T days that is no greater than O( √ T ).